Image display method, image display apparatus, and storage medium

ABSTRACT

An image display method includes acquiring a first image of a fundus within a first area of an eye to be inspected, acquiring interference signal sets corresponding to a plurality of frames, which are acquired with an intention to acquire the same cross section, for a plurality of different cross sections, generating, based on the interference signal sets corresponding to the plurality of frames, a motion contrast image of the fundus within a second area included in the first area, and superimposing, for display, information acquired from a portion of the motion contrast image of the fundus onto a corresponding position of the first image of the fundus.

BACKGROUND

Field

This disclosure relates to an image display method, an image displayapparatus, and a storage medium.

Description of the Related Art

As a method of acquiring a tomographic image of an object to bemeasured, e.g., a living body, in a non-destructive and non-invasivemanner, optical coherence tomography (hereinafter referred to as “OCT”)has been put into practical use. OCT is widely used particularly in thefield of ophthalmology in order to acquire a tomographic image of aretina in a fundus of an eye to be inspected for ophthalmologicdiagnosis of the retina, or the like.

In OCT, light reflected from the object to be measured and lightreflected from a reference mirror are caused to interfere with eachother, and time dependence or wavenumber dependence of an intensity ofthe interference light is analyzed, to thereby acquire a tomographicimage. As apparatus for acquiring such an optical coherence tomographicimage, there are known a time domain OCT apparatus, a spectral domainOCT apparatus, and a swept source OCT apparatus. The time domain OCTapparatus is configured to acquire depth information on the object to bemeasured by changing a position of the reference mirror. The spectraldomain optical coherence tomography (SD-OCT) apparatus using a broadbandlight source is configured to split interference light into light beamshaving different wavelengths with a spectroscope to acquire depthinformation on the object to be measured. The swept source opticalcoherence tomography (SS-OCT) apparatus is configured to use awavelength-tunable light source apparatus capable of changing anoscillation wavelength. The SD-OCT and the SS-OCT are collectivelyreferred to as “Fourier domain optical coherence tomography (FD-OCT)”.

In recent years, there has been proposed simulated angiography usingFD-OCT, which is referred to as “OCT angiography (OCTA)” (Makita et al.,“Optical Coherence Angiography,” Optics Express, 14(17), 7821-7840(2006)). In fluorescence angiography, which is general angiography incontemporary clinical medicine, injection of a fluorescent dye (e.g.,fluorescein or indocyanine green) into a body is required, and a vesselthrough which the fluorescent dye passes is displayed two-dimensionally.Meanwhile, OCTA enables non-invasive and simulated imaging of vessels,and enables three-dimensional display of a network of a blood flowregion. Further, OCTA is attracting attention because OCTA provides ahigher resolution as compared with fluorescence angiography, whichenables a minute vessel or blood flow of the fundus to be drawn.

There have been proposed a plurality of methods for OCTA, which differin blood flow detection method. For example, in “An et al. ‘Opticalmicroangiography provides correlation between microstructure andmicrovasculature of optic nerve head in human subjects,’ J. Biomed. Opt.17, 116018 (2012)”, there is proposed a method involving extracting onlya signal that is changing in time from OCT signals, to thereby obtain anOCT signal due to a blood flow. In “Fingler et al. ‘Mobility andtransverse flow visualization using phase variance contrast withspectral domain optical coherence tomography’ Optics Express. Vol. 15,No. 20. pp. 12636-12653 (2007)”, there is proposed a method using aphase variance due to a blood flow. In each of “Mariampillai et al.,‘Optimized speckle variance OCT imaging of microvasculature,’ OpticsLetters 35, 1257-1259 (2010)” and U.S. Patent Application PublicationNo. 2014/221827, there is proposed a method using an intensity variancedue to a blood flow.

However, in the above-mentioned OCTA, a blood flow region can beacquired in detail, and hence, an inspector may have difficulty inidentifying a connection of a specific vessel of interest and how thespecific vessel extends. Further, an image relating to the blood flowregion acquired through the above-mentioned OCTA and an image relatingto the structure information acquired through OCT are acquired asindependent and separate images. Thus, in actual diagnosis, theinspector needs to compare those images with each other alternately.However, because the detailed blood flow region acquired through OCTA isdisplayed in detail, its correspondence to structure information on theeye to be inspected that is acquired from an OCT intensity image, afundus photograph, or the like, is difficult to understand.

This disclosure has been made in view of the above-mentionedcircumstances, and it is an object of this disclosure to facilitateunderstanding of correspondence between an image relating to a bloodflow region acquired through OCTA and structure information acquiredthrough OCT, or the like.

In order to solve the above-mentioned problem, one aspect of the presentinvention is an image display method comprising the steps of acquiring afirst image within a first area of an object to be inspected, acquiringinterference signal sets corresponding to a plurality of frames, whichare acquired with an intention to acquire the same cross section, for aplurality of different cross sections, generating, based on theinterference signal sets corresponding to the plurality of frames, amotion contrast image within a second area included in the first area,and superimposing, for display, information acquired from a part of themotion contrast image onto a corresponding position of the first image.

According to this disclosure, understanding of the correspondencebetween the image relating to the blood flow region acquired throughOCTA and the structure information acquired through OCT, or the like,can be facilitated.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram for illustrating an overview of an entireconfiguration of an apparatus according to a first embodiment of thisdisclosure.

FIG. 2 is a diagram for illustrating an example of a scan patternaccording to the first embodiment.

FIG. 3 is a flowchart for illustrating an entire processing procedurefor image generation according to the first embodiment.

FIG. 4 is a flowchart for illustrating an interference signalacquisition procedure according to the first embodiment.

FIG. 5 is a flowchart for illustrating a signal processing procedureaccording to the first embodiment.

FIG. 6 is a flowchart for illustrating a three-dimensional blood flowregion information acquisition and display procedure according to thefirst embodiment.

FIG. 7A and FIG. 7B are each an explanatory graph for showing asegmentation result according to the first embodiment.

FIG. 8 is a diagram for illustrating an example of a display screenaccording to the first embodiment.

FIG. 9 is a diagram for illustrating an example of a display methodaccording to the first embodiment.

FIG. 10 is a diagram for illustrating an example of a depth selectionmethod according to a second embodiment of this disclosure.

FIG. 11A and FIG. 11B are each an explanatory diagram for illustratingan example of a method of generating a two-dimensional intensity imageaccording to the second embodiment.

FIG. 12 is a diagram for illustrating an example of a method ofdisplaying a selected region according to the second embodiment.

FIG. 13 is a flowchart for illustrating an example of an area divisionprocessing procedure.

FIG. 14A and FIG. 14B are each a diagram for illustrating an example ofa display method of displaying two images.

FIG. 15 is a flowchart for illustrating an example of a comparisonprocedure to be performed when a vessel is estimated.

FIG. 16 is a diagram for illustrating an example of a method ofdisplaying an analysis result of the vessel estimation.

FIG. 17 is a diagram for illustrating an example of a display screenaccording to a third embodiment of this disclosure.

FIG. 18A and FIG. 18B are each a diagram for illustrating an example ofa display screen according to a fourth embodiment of this disclosure.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of this disclosure are described below in detail withreference to the drawings. The following embodiments are not to limitthis disclosure laid down in the appended claims, and not allcombinations of features described in the following embodiments areindispensable to the solutions of this disclosure.

Further, herein, an OCT signal displayed as an image is referred to as“intensity image”. Further, a signal that is changing in time among theOCT signals and is displayed as an image is referred to as “motioncontrast image”, a pixel value of the motion contrast image is referredto as “motion contrast”, and a data set of the motion contrast image isreferred to as “motion contrast data”.

(First Embodiment)

In a first embodiment of this disclosure, an example is described inwhich a tomographic image is generated from a three-dimensional opticalinterference signal acquired through imaging, and a motion contrast iscalculated to acquire three-dimensional blood flow region information.

[Configuration of Entire Image Forming Apparatus]

FIG. 1 is a diagram for illustrating a configuration example of an imageforming method and apparatus using optical coherence tomographyaccording to an embodiment of this disclosure. The image forming methodand apparatus include an OCT apparatus 100 serving as an opticalcoherence tomography acquisition unit configured to acquire an opticalcoherence tomography signal, and a control portion 143. For example, theSD-OCT apparatus and the SS-OCT apparatus described above are applicableto this disclosure as the OCT apparatus. In the embodiment describedbelow, a configuration of a case where the OCT apparatus is the SS-OCTapparatus is described.

<Configuration of OCT Apparatus 100>

Now, a configuration of the OCT apparatus 100 is described withreference to FIG. 1.

A light source 101 is a swept source (hereinafter referred to as “SS”)light source, and is configured to emit light while sweeping awavelength of the light with a sweeping central wavelength of 1,050 nmand a sweeping width of 100 nm, for example.

Light emitted from the light source 101 is guided to a beam splitter 110via an optical fiber 102, and the light is divided into measuring light(also referred to as “OCT measuring light”) and reference light (alsoreferred to as “reference light corresponding to OCT measuring light”).The beam splitter 110 divides the light at a ratio of 90 (referencelight):10 (measuring light). The measuring light obtained by division isemitted to a measurement optical path via an optical fiber 111. On themeasurement optical path, in order from the optical fiber 111 to an eyeto be inspected 118, a collimator lens 112, a galvano scanner 114, ascan lens 115, and a focus lens 116 are arranged.

The measuring light emitted to the measurement optical path is formedinto collimated light by the collimator lens 112. The measuring lightformed into the collimated light enters the eye to be inspected 118 viathe galvano scanner 114, the scan lens 115, and the focus lens 116,which are configured to scan the measuring light on a fundus Er of theeye to be inspected 118. The galvano scanner 114 is described here as asingle mirror, but is actually formed of two galvano scanners (notshown), e.g., an X-axis scanner and a Y-axis scanner, so as toraster-scan the fundus Er of the eye to be inspected 118.

The focus lens 116 is fixed onto a stage 117, and when the stage 117moves in an optical axis direction, focus can be adjusted with the focuslens 116. The galvano scanner 114 and the stage 117 are controlled by asignal acquisition control portion 145 described later, to thereby beable to scan the measuring light within a desired range of the fundus Erof the eye to be inspected 118 (also referred to as “acquisition rangeof tomographic image”, “acquisition position of tomographic image”, and“irradiation position of measuring light”).

Although not described in detail in this embodiment, it is desired thatthe OCT apparatus 100 be provided with a tracking function of detectingmovement of the fundus Er to cause the mirrors of the galvano scanner114 to scan the light while following the movement of the fundus Er. Ageneral technology can be used to perform a tracking method, and thetracking method may be performed in real time, or may be performed inpost processing.

As a method of acquiring a fundus image in order to detect the movementof the fundus Er, there is given, for example, a method using a scanninglaser ophthalmoscope (SLO). In this method, an image of the fundus Erwithin a plane perpendicular to an optical axis (fundus surface image)is acquired over time through use of SLO to extract a characteristicportion within the image, e.g., a portion in which a vessel branches.How the characteristic portion within the acquired two-dimensional imagehas moved is calculated as a moving amount of the fundus Er, and thecalculated moving amount is fed back to the galvano scanner 114. In thismanner, real-time tracking can be performed.

As described above, the measuring light enters the eye to be inspected118 via the focus lens 116 fixed onto the stage 117, and is focused ontothe fundus Er by the focus lens 116. The measuring light that hasirradiated the fundus Er is reflected and scattered by each retinallayer and returns to the beam splitter 110 via the measurement opticalpath. The return light of the measuring light that has entered the beamsplitter 110 passes through an optical fiber 126 to enter a beamsplitter 128.

Meanwhile, the reference light obtained by division by the beam splitter110 is emitted to a reference optical path via an optical fiber 119 a, apolarization controller 150, and an optical fiber 119 b. On thereference optical path, in order from the optical fiber 119 b, acollimator lens 120, a dispersion compensation glass 122, an ND filter123, and a collimator lens 124 are arranged.

The reference light emitted from the optical fiber 119 b is formed intocollimated light by the collimator lens 120. The polarization controller150 is capable of changing polarization of the reference light to adesired polarization state. The reference light enters an optical fiber127 via the dispersion compensation glass 122, the ND filter 123, andthe collimator lens 124. One end of the collimator lens 124 and one endof the optical fiber 127 are fixed onto a coherence gate stage 125, andthe collimator lens 124 and other members are controlled by the signalacquisition control portion 145 described later so as to be driven in anoptical axis direction depending on the difference in axial length amongsubjects to be examined, or the like. Although an optical path length ofthe reference light is changed in this embodiment, an optical pathlength of the measuring light may be changed as long as the differencein length between the optical path of the measuring light and theoptical path of the reference light can be changed.

The reference light that has passed through the optical fiber 127 entersthe beam splitter 128. In the beam splitter 128, the above-mentionedreturn light of the measuring light and the reference light are coupledto become interference light, and the interference light is divided intotwo light beams. The interference light is divided into interferencelight beams having phases inverted to each other (hereinafter expressedas “positive component” and “negative component”). The positivecomponent of the interference light obtained by division passes throughan optical fiber 129 to enter one of input ports of a detector 141.Meanwhile, the negative component of the interference light passesthrough an optical fiber 130 to enter the other input port of thedetector 141. The detector 141 serves as a differential detector, andwhen two interference signals having phases inverted to each other by180° are input, removes a DC component and outputs only an interferencecomponent.

The interference signals detected by the detector 141 are output as anelectrical signal corresponding to intensity of light, and theelectrical signal is input to a signal processing portion 144, which isan example of a tomographic image generation portion.

<Configuration of Control Portion 143>

The control portion 143 for controlling the image forming apparatus isdescribed.

The control portion 143 includes the signal processing portion 144, thesignal acquisition control portion 145, a display portion 146, and adisplay control portion 149. Further, the signal processing portion 144includes an image generation portion 147 and a map generation portion148. The image generation portion 147 has a function of generating aluminance image and a motion contrast image based on the electric signalsent thereto, and the map generation portion 148 has a function ofgenerating layer information (segmentation of a retina) based on theluminance image.

The signal acquisition control portion 145 is configured to control thestage 117, the coherence gate stage 125, and other members as describedabove. The signal processing portion 144 is configured to generate animage, analyze the generated image, and generate visible information onan analysis result based on the signal output from the detector 141.

The image generated by the signal processing portion 144 and itsanalysis result are sent to the display control portion 149, and thedisplay control portion 149 displays the sent image and analysis resulton a display screen of the display portion 146. The display portion 146is a display, e.g., a liquid crystal display. Image data generated bythe signal processing portion 144 may be transmitted to the displayportion 146 in a wired or wireless manner after being sent to thedisplay control portion 149. Further, although the display portion 146and other portions are included in the control portion 143 in thisembodiment, this disclosure is not limited thereto. The display portion146 and other portions may be provided separately from the controlportion 143, and may be, for example, a tablet computer, which is anexample of a device that can be carried around by a user. In this case,it is preferred that the display portion be provided with a touch panelfunction so that movement of a display position of an image,enlargement/reduction of the image, change of an image to be displayed,other such operations can be performed on a touch panel.

[Scan Pattern]

<Mode of Scanning Measuring Light in OCT Apparatus>

Now, a mode of scanning the measuring light in the OCT apparatus isdescribed. As described above, the interference light is acquired basedon the return light of the measuring light that has been radiated ontoan arbitrary point on the fundus of the eye to be inspected 118 and thecorresponding reference light. The signal processing portion 144processes the electrical signal corresponding to the intensity of theinterference light detected by the detector 141 to acquire image data atthe arbitrary point in a depth direction. A process of acquiringinformation on a layer at a given point on the eye to be inspected 118has been described above. Acquisition of the information on a layer inthe depth direction of the eye to be inspected 118 is referred to as“A-scan”.

Further, acquisition of the information on a layer of the eye to beinspected 118 in a direction orthogonal to that of the A-scan, that is,acquisition of information on a plurality of layers in theabove-mentioned depth direction in a scan direction for acquiring atwo-dimensional image is referred to as “B-scan”. Still further,acquisition of information on the plurality of layers in the depthdirection in a scan direction orthogonal to both of the scan directionsof the A-scan and the B-scan is referred to as “C-scan”. When atwo-dimensional raster scan is performed on a fundus surface in order toacquire a three-dimensional tomographic image, a direction of ahigh-speed scan is referred to as “B-scan direction”, and a direction ofa low-speed scan in which scans are performed while lining up B-scans ina direction orthogonal to the B-scan direction is referred to as “C-scandirection”. A two-dimensional tomographic image is acquired byperforming the A-scan and the B-scan, and the three-dimensionaltomographic image is acquired by performing the A-scan, the B-scan, andthe C-scan. The B-scan and the C-scan are performed by changing theirradiation position of the measuring light with the above-mentionedgalvano scanner 114.

As described above, the galvano scanner 114 is formed of the X-axisscanner and the Y-axis scanner (not shown), each of which is formed of adeflecting mirror arranged to have a rotational axis orthogonal to thatof the other deflecting mirror. For example, the X-axis scanner isconfigured to scan the measuring light on the fundus Er in an X-axisdirection, and the Y-axis scanner is configured to scan the measuringlight on the fundus Er in a Y-axis direction. The X-axis direction andthe Y-axis direction are directions that are perpendicular to an axialdirection of an eyeball and are perpendicular to each other.

Further, a line scan direction for the B-scan or the C-scan and theX-axis direction or the Y-axis direction do not need to match.Accordingly, the line scan direction for the B-scan or the C-scan may bedetermined appropriately depending on a two-dimensional orthree-dimensional tomographic image desired to be taken.

<Mode of Scanning Measuring Light According to This Embodiment>

In OCTA, in order to measure a change with time of the OCT interferencesignal due to a blood flow, measurement needs to be performed aplurality of times at the same position. In this embodiment, whilerepeating the B-scan (scan in the X-axis direction) at the same positionm times, the OCT apparatus performs a scan of moving a scan position ton y-positions (positions to which the irradiation position of themeasuring light is moved in the Y-axis direction). A specific scanpattern is illustrated in FIG. 2. As illustrated in FIG. 2, in thisembodiment, the OCT apparatus repeats the B-scan m times at each of then y-positions y1 to yn on a fundus plane.

In this case, as the value of m becomes larger, the number of times ofmeasurement at the same position also increases, and hence an accuracyof detecting the blood flow increases. Meanwhile, the scan timeincreases, and hence there arise problems in that a motion artifactoccurs in an image due to movement of an eye (involuntary eye movementduring fixation) during a scan and in that burden on the subject to beexamined increases. In this embodiment, m is set to 4 in considerationof the balance between the two. The control portion 143 may beconfigured to automatically change m depending on an A-scan speed of theOCT apparatus and motion analysis of a fundus surface image of the eyeto be inspected 118.

Further, in FIG. 2, p represents the number of samples of the A-scan inone B-scan. In other words, in one B-scan, the A-scan is performed ateach of positions x1 to xp, and the size of the plane image isdetermined based on p×n. As the value of p×n becomes larger, a widerrange can be scanned as long as a measurement pitch is the same.However, the scan time increases, and hence the above-mentioned problemsin the motion artifact and the increase in the burden on the subject tobe examined arise. In this embodiment, n and p are each set to 300 inconsideration of the balance between the two. The values of n and p maybe freely changed as necessary.

Further, Δx of FIG. 2 represents an interval (x-pitch) between adjacentx-positions, and Δy of FIG. 2 represents an interval (y-pitch) betweenadjacent y-positions. In this embodiment, each of the x-pitch and they-pitch is determined as ½ of a beam spot diameter of the light radiatedto the fundus, specifically, 10 μm. When each of the x-pitch and they-pitch is set to ½ of the beam spot diameter on the fundus, the imageto be generated can be formed with high resolution. Even when each ofthe x-pitch and the y-pitch is set to be smaller than ½ of the beam spotdiameter on the fundus, an effect of further increasing the resolutionof the image to be generated is small.

In contrast, when each of the x-pitch and the y-pitch is set to belarger than ½ of the beam spot diameter on the fundus, the resolutiondeteriorates, but the image of a wider range can be acquired with asmall data size. However, each of the x-pitch and the y-pitch may befreely changed depending on a clinical requirement.

A scan range in this embodiment is p×Δx=3 mm in the X-axis direction andn×Δy=3 mm in the Y-axis direction.

<Mode of Acquiring OCT Interference Signal and OCTA Signal>

The OCT interference signal and an OCTA signal may be acquired in thesame step, or may be acquired in different steps. In the following, acase when the OCT interference signal and the OCTA signal are acquiredin the same step is described. When those signals are acquired in thesame step, the signal acquisition control portion 145 measures the sameposition a plurality of times during the scan with the measuring light,to thereby acquire interference signal sets corresponding to a pluralityof frames to be used when forming the three-dimensional tomographicimage. In this case, the signal acquisition control portion 145functions as a signal acquiring unit. The interference signal set is aset of interference signals acquired through the above-mentioned oneB-scan, and means a set of interference signals from which one frame ofthe tomographic image of the B-scan can be formed. Further, the eye tobe inspected performs involuntary eye movement during fixation at anytime, and hence even when an attempt is made to form an image of thesame cross section of the eye to be inspected by acquiring theinterference signal sets corresponding to the plurality of framesthrough a plurality of times of scans at the same position on the eye tobe inspected, it is difficult to accurately acquire the same crosssection in actual cases. Accordingly, the interference signal setscorresponding to the plurality of frames described here are understoodas the interference signal sets corresponding to the plurality offrames, which are acquired with an intention to acquire the same crosssection. As another example, the above-mentioned interference signal setis the interference signal sets corresponding to the plurality of framesacquired by scanning, with a scan unit exemplified by the galvanoscanner 114 configured to scan the measuring light on the fundus, themeasuring light repeatedly along the same scan line (B-scan line).

The signal processing portion 144 processes the interference signal setsto calculate the OCT interference signal and the OCTA signal. Throughuse of the interference signal sets acquired in the same step, thepixels of the OCT intensity image and the OCTA motion contrast image canbe formed at the same position. In other words, in this embodiment, theinterference signal set to be used for generating the OCT intensityimage is included in the interference signal sets corresponding to theplurality of frames, which are acquired in order to generate the OCTAmotion contrast image. Further, the OCT intensity image to be displayedas a two-dimensional image may be acquired by averaging (superimposing)two or more interference signal sets among the above-mentioned acquiredinterference signal sets corresponding to the plurality of frames, thatis, two frames. With this configuration, random noises contained in theinterference signal sets are averaged, and hence the noise of theintensity image can be reduced. In this case, the signal processingportion 144 functions as a motion contrast image generation unitconfigured to generate the motion contrast image through use ofcorresponding pieces of pixel data of the frames in the interferencesignal sets corresponding to the plurality of frames forming the samecross section.

Next, a case when the OCT interference signal and the OCTA signal areacquired in different steps is described. When those signals areacquired in different steps, for example, the signal acquisition controlportion 145 causes the OCT apparatus 100 to acquire the OCTA signalafter acquiring the OCT interference signal. In a scan pattern at thetime of acquisition of the OCT interference signal, the number of timesof repetition, the pitch, and other values that are suited to the scanpattern are used. For example, the OCT interference signal does notnecessarily need to be acquired by repeating the B-scan m times.Further, the scan range does not need to be the same for both of theacquisition of the OCT interference signal and the acquisition of theOCTA signal. Therefore, as long as the scan time is the same, the OCTinterference signal can be acquired with a wider range as a firstpredetermined range. Further, when the signals are acquired in differentsteps, for example, those signals can be used in the following manner:while a wide range is observed with the OCT three-dimensionaltomographic image, the OCTA signal is acquired in detail only in apredetermined area of interest as a second predetermined range includedin the first predetermined range.

Next, referring to a flowchart of FIG. 3, a specific processingprocedure of the image forming method or the image display methodaccording to this embodiment is described. A detailed description ofeach processing step illustrated in the flow is given later. Referringto FIG. 3, an example in which the OCT interference signal and the OCTAsignal are acquired in different steps is described.

In Step S101, which is a first signal acquisition step, that is, a stepof acquiring an object-to-be-inspected image to be displayed as atwo-dimensional image, the signal acquisition control portion 145controls the OCT apparatus 100 to acquire the optical coherencetomography signal.

In Step S102, which is a second signal acquisition step, that is, asignal acquisition step, the signal acquisition control portion 145controls the OCT apparatus 100 to acquire the optical coherencetomography signals corresponding to the plurality of frames. Morespecifically, the interference signal sets corresponding to theplurality of frames, which are based on the measuring light controlledto scan the same position in a sub-scan direction a plurality of times,are acquired. Further, this operation is executed a plurality of timeswhile shifting the scan position in the sub-scan direction in order toacquire the interference signal sets from a plurality of different crosssections, to thereby acquire the interference signal sets sufficient forforming a three-dimensional tomographic image.

In Step S103, which is an intensity image (display information)generation step, the control portion 143 calculates three-dimensionaltomographic image data of the eye to be inspected 118 based on theoptical coherence tomography signal acquired in the first signalacquisition step, to thereby generate an intensity image. Further, thecontrol portion 143 calculates three-dimensional motion contrast databased on the optical coherence tomography signals corresponding to theplurality of frames acquired in the second signal acquisition step, tothereby generate a three-dimensional motion contrast image.

In Step S104, which is a display generation step, the control portion143 generates and displays display information based on the intensityimage and the three-dimensional motion contrast image. In response to aninspector's operation, the control portion 143 reconstructs andredisplays the display information. A detailed description of thereconstruction processing is given later.

After the above-mentioned steps are executed, the processing procedureof the image forming method according to this embodiment is brought toan end.

That is, in this embodiment, in Step S101, which is the first signalacquisition step, a surface image of the fundus (first image) in a firstarea of the fundus Er (object to be inspected) of the eye to beinspected 118 is acquired. In this embodiment, the first image isgenerated based on the interference signal set acquired from the firstarea. At this time, the interference signal set may be selected for usefrom among the interference signal sets corresponding to the pluralityof frames acquired in Step S102. In this case, as described later, thefirst image is generated by superimposing, in the depth direction,pieces of data of the three-dimensional tomographic image of the objectto be inspected acquired from the interference signal sets correspondingto the plurality of frames.

In Step S102, which is the second signal acquisition step, theinterference signal sets corresponding to the plurality of frames, whichare acquired with an intention to acquire the same cross section, areacquired for a plurality of different cross sections. From the acquiredinterference signal sets corresponding to the plurality of frames, inStep S103, which is the display information generation step, the motioncontrast image based on those signals in a second area included in thefirst area on the fundus Er is generated. Further, in Step S104, whichis the display generation step, information acquired from a part of themotion contrast image is superimposed onto a corresponding position ofthe first image exemplified by the above-mentioned fundus surface image,and the resultant image is displayed.

It is preferred that, in the superimposition of images, positionalignment be performed between data of the three-dimensional tomographicimage of the object to be inspected and three-dimensional image datathat is based on the interference signal set used for generating themotion contrast image. In this case, as described in this embodiment,information on vessels of the object to be inspected can be given as theinformation acquired from a part of the motion contrast image, andexamples of the information include a state representing a connectionstate of vessels as described later. Further, in this case, theinformation on vessels may be extracted not only from the motioncontrast image but also from the first image, and the extractedinformation on vessels may be superimposed for display onto acorresponding position of one of the images from which the informationis not extracted.

[Interference Signal Acquisition Procedure]

Next, referring to FIG. 4, a specific processing procedure of Step S101and Step S102, which are the first signal acquisition step and thesecond signal acquisition step according to this embodiment,respectively, is described.

In Step S109, the signal acquisition control portion 145 sets an index iof a position yi of FIGS. 2 to 1.

In Step S110, the OCT apparatus 100 drives a scan position to theposition yi. In Step S119, the signal acquisition control portion 145sets an index j of a repetitive B-scan to 1. In Step S120, the OCTapparatus 100 performs the B-scan.

In Step S130, the detector 141 detects the interference signal for eachA-scan, and the detected interference signal is stored in the signalprocessing portion 144 via an A/D converter (not shown). The signalprocessing portion 144 acquires p samples of the interference signals ofthe A-scan, to thereby acquire the interference signals corresponding toone B-scan.

In Step S139, the signal acquisition control portion 145 increments theindex j of the repetitive B-scan.

In Step S140, the signal acquisition control portion 145 determineswhether or not j is larger than a predetermined number of times (m). Inother words, the signal acquisition control portion 145 determineswhether or not the B-scan at the position yi has been repeated m times.When the B-scan at the position yi has not been repeated m times, theflow returns to Step S120, and the measurement of the B-scan at the sameposition is repeated. When the B-scan at the position yi has beenrepeated m times, the flow proceeds to Step S149. In Step S149, thesignal acquisition control portion 145 increments the index i of theposition yi. In Step S150, the signal acquisition control portion 145determines whether or not i is larger than a predetermined number oftimes of y-position measurement (n), that is, determines whether or notthe B-scan has been performed at all of the n y-positions. When themeasurement has not been performed by the predetermined number of timesof y-position measurement (no), the flow returns to Step S110, and themeasurement at the next measurement position is repeated. When themeasurement has been performed by the predetermined number of times ofy-position measurement (yes), the flow proceeds to the next Step S160.In Step S101 and Step S102 described above, which are the first signalacquisition step and the second signal acquisition step, n is set to 1and m is set to 4.

In Step S160, the OCT apparatus 100 acquires background data. The OCTapparatus 100 executes the A-scan 100 times under a state in which ashutter (not shown) is closed, and the signal acquisition controlportion 145 averages signals acquired through one hundred times of theA-scan and stores the resultant signal. The number of times ofmeasurement of the background is not limited to one hundred times asexemplified above.

After the execution of the above-mentioned processing steps, theinterference signal acquisition procedure according to this disclosureis brought to an end.

In acquisition of an OCTA interference signal, by setting the number oftimes of measurement and the position to values suited to theacquisition of the OCTA interference signal, the OCTA interferencesignal can be acquired with a procedure similar to that of theacquisition of the OCT signal.

In this embodiment, in order to acquire the OCT signals and form thethree-dimensional tomographic image, an object-to-be-inspected image tobe displayed two-dimensionally is generated based on data acquired fromthe interference signal set formed of the OCT signals. As describedabove, the range from which the OCT signal is acquired does not need tobe the same as the range from which the OCTA signal is acquired. Thus,as described above, this object-to-be-inspected image is generated as animage within the first area including the second area of the eye to beinspected 118 from which the OCTA signal is acquired.

[Signal Processing Procedure]

Next, referring to FIG. 5, specific processing of generating displayinformation including three-dimensional image information of OCT andthree-dimensional blood flow region information of OCTA, which isperformed in Step S103 of FIG. 3 according to this embodiment, isdescribed. In this disclosure, in order to generate thethree-dimensional blood flow region information based on the OCTAinformation, the motion contrast of OCTA is calculated.

In Step S210, the signal processing portion 144 sets the index i of theposition yi to 1. In Step S220, the signal processing portion 144extracts the interference signals of the repetitive B-scan(corresponding to m times of the B-scan) at the position yi (i=1 in thiscase). In Step S230, the signal processing portion 144 sets the index jof the repetitive B-scan to 1. In Step S240, the signal processingportion 144 extracts j-th (j=1 in this case) B-scan data.

In Step S250, the signal processing portion 144 performs reconstructionprocessing on the interference signal of the B-scan data extracted inStep S240, to thereby generate a luminance image of the tomographicimage. First, the image generation portion 147 removes from theinterference signal a fixed pattern noise formed of the background data.The fixed pattern noise is removed by averaging A-scan signals of thedetected plurality of pieces of background data to extract a fixedpattern noise and subtracting the fixed pattern noise from the inputinterference signal. Next, in order to optimize a depth resolution and adynamic range, which have a trade-off relationship when Fouriertransform is performed at a finite interval, the image generationportion 147 performs a desired window function. After that, the imagegeneration portion 147 performs FFT to generate the luminance image ofthe tomographic image.

In Step S260, the signal processing portion 144 increments the index jof the repetitive B-scan. In Step S270, the signal processing portion144 determines whether or not the incremented j is larger than m. Inother words, the calculation of the luminance of the B-scan at theposition yi has been repeated m times. When the determination results in“no”, the flow returns to Step S240, and the calculation of theluminance of the repetitive B-scan at the same y-position is repeated.When the determination results in “yes”, the flow proceeds to the nextstep.

In Step S280, the signal processing portion 144 performs the positionalignment on m frames of the repetitive B-scan at the given position yi.Specifically, first, the signal processing portion 144 selects fromamong the m frames an arbitrary one frame as a template. The frame to beselected as the template may be selected by calculating a correlationfor every combination of frames, calculating a sum of correlationcoefficients for each frame, and selecting one of the frames having thelargest sum. Next, the template is compared with each of the frames tocalculate a positional deviation amount (δX, δY, δθ) of each of theframes. Specifically, a normalized cross-correlation (NCC), which is amark indicating a similarity, is calculated while changing the positionand angle of the template image, and the difference between imagepositions at the time when the NNC is the largest is acquired as thepositional deviation amount.

In this disclosure, the mark indicating the similarity may be changed tovarious scales as long as the scale indicates a similarity betweenfeatures of images within the template and the frame. For example, a sumof absolute difference (SAD), a sum of squared difference (SSD), azero-means normalized cross-correlation (ZNCC), a phase only correlation(POC), or a rotation invariant phase only correlation (RIPOC) may alsobe used.

Next, the signal processing portion 144 applies position correction tom-1 frames other than the template depending on the positional deviationamount (δX, δY, δθ), to thereby perform the position alignment of mframes. Further, when the three-dimensional image information of OCT andthe three-dimensional blood flow region information of OCTA are acquiredin different steps as in this embodiment, position alignment is alsoperformed between the three-dimensional image information and thethree-dimensional blood flow region information. In other words, theposition alignment is performed between data of the three-dimensionaltomographic image calculated based on the interference signal setacquired for the purpose of generating the OCT intensity image and dataof the three-dimensional tomographic image calculated based on theinterference signal set acquired for the purpose of generating themotion contrast image. Through execution of the position alignmentbetween those pieces of data, understanding of correspondence betweenthe three-dimensional image information and the three-dimensional bloodflow region information can be facilitated. The above-mentioned positionalignment may be performed by a method similar to that of the positionalignment between the frames.

In Step S290, the signal processing portion 144 averages the luminanceimage subjected to the position alignment, which is calculated in StepS280, to thereby generate an averaged luminance image.

In Step S300, the map generation portion 148 executes segmentation(region information acquisition) of the retina from the averagedluminance image generated in Step S290 by the signal processing portion144. In the first embodiment described here, this step is not used, andhence this step is skipped. This step is described in a secondembodiment of this disclosure.

In Step S310, the image generation portion 147 calculates a motioncontrast. In this embodiment, a variance is calculated for each pixel atthe same position from the luminance image of each of the m frames oftomographic images output in Step S250 by the signal processing portion144, and the calculated variance is determined as the motion contrast.

There are various types of methods of calculating the motion contrast.For example, the type of characteristic amount to be used as the motioncontrast in this disclosure is a change in luminance value of each pixelof a plurality of B-scan images at the same y-position. Therefore, anymark that indicates this change in luminance value may be applied tocalculate the motion contrast. Further, instead of using the variancefor each pixel at the same position, which is acquired from theluminance image of each of the m frames of the tomographic image, themotion contrast may be acquired by another method. For example, acoefficient of variation normalized with an average value of the samepixels of the respective frames may also be used.

In Step S320, the signal processing portion 144 executes first thresholdprocessing on the motion contrast output by the signal processingportion 144. A first threshold value is calculated based on the averagedluminance image output in Step S290 by the signal processing portion144. Specifically, an area in which only random noise is displayed in anoise floor is extracted from the averaged luminance image to calculatea standard deviation σ, and “(average luminance in noise floor)+2σ” isset as the first threshold value. The signal processing portion 144 setsthe value of the motion contrast corresponding to an area in which eachluminance is equal to or smaller than the first threshold value to 0.

Through the first threshold processing of Step S320, noise can bereduced by removing the motion contrast originating from a change inluminance due to the random noise.

As the first threshold value becomes smaller, sensitivity of detectingthe motion contrast increases, whereas a noise component also increases.In contrast, as the first threshold value becomes larger, noisedecreases, whereas the sensitivity of detecting the motion contrast alsodecreases. In this embodiment, the threshold value is set to “(averageluminance in noise floor)+2σ”, but the threshold value is not limitedthereto.

In Step S330, the signal processing portion 144 increments the index iof the position yi.

In Step S340, the signal processing portion 144 determines whether ornot i is larger than n. In other words, the signal processing portion144 determines whether or not the position alignment, the calculation ofthe averaged luminance image, the calculation of the motion contrast,and the threshold processing have been performed at all of the ny-positions.

When the determination results in “no”, the flow returns to Step S220.When the determination results in “yes”, the flow proceeds to the nextStep S350.

At the time when Step S340 is finished, three-dimensional motioncontrast data (three-dimensional data) on each pixel of the B-scanimages (data in Z-axis (depth) and X-axis directions) has been acquiredat all of the y-positions. In Step S350, the signal processing portion144 performs display information generation processing based on thosepieces of three-dimensional image information.

Now, the display information generation processing according to thisembodiment is described. FIG. 6 is a flowchart for illustrating detailsof the processing executed in Step S350.

In the generation of the display information, in Step S351, the signalprocessing portion 144 acquires the three-dimensional motion contrastdata acquired in the above-mentioned processing.

In Step S352, in order to remove noise without removing the blood flowregion information, the signal processing portion 144 performs smoothingprocessing on the three-dimensional motion contrast data. Althoughoptimal smoothing processing differs depending on a characteristic ofthe motion contrast, the following smoothing methods are conceivable,for example:

-   -   a smoothing method of outputting a maximum value of the motion        contrasts from nx×ny×nz voxels near a pixel of interest;    -   a smoothing method of outputting an average value of the motion        contrasts of the nx×ny×nz voxels near the pixel of interest;    -   a smoothing method of outputting a median value of the motion        contrasts of the nx×ny×nz voxels near the pixel of interest;    -   a smoothing method of weighting with a weight that is based on a        distance with regard to the motion contrasts of the nx×ny×nz        voxels near the pixel of interest;    -   a smoothing method of weighting with a weight that is based on        the distance and a weight that is based on the difference in        pixel value from the pixel of interest with regard to the motion        contrasts of the nx×ny×nz voxels near the pixel of interest; and    -   a smoothing method of outputting a value using a weight that is        based on a similarity between a motion contrast pattern of a        small area around the pixel of interest and a motion contrast        pattern of a small area around a surrounding pixel.

Another method of performing smoothing without removing the blood flowregion information may also be used.

After the above-mentioned smoothing processing, in Step S353, the signalprocessing portion 144 acquires, from the display control portion 149,initial values of a threshold value to be used when a pixel to bedisplayed is determined in Step S354 and a range of display in the depthdirection. The initial value of the display range is normally set toabout ¼ in the depth direction, and is set to such a position as toinclude most of the range of a retinal surface layer. The initial valueof the display range in this case is not set to the entire range in thedepth direction because a network of main vessels and capillary vesselsin a surface layer portion is desired to be displayed first in aneasy-to-view manner. In other words, when the surface layer portionincluding the network of main vessels and capillary vessels and an RPElayer, which does not have vessels and contains a large amount of noise,are displayed simultaneously, such display interferes withidentification of the network of main vessels and capillary vessels inthe surface layer portion. Regarding the initial value of the displaythreshold value, it suffices that a representative value is acquired inadvance from a healthy eye, or the like, to be set as the initial value.As another example, when imaging is performed repeatedly, the lastimaging data may be used. Further, the display threshold value may beadjusted in a processing step described later.

Next, in Step S354, display threshold processing is performed in whichthe initial value of the display threshold value is used to display apixel of the three-dimensional data subjected to the smoothingprocessing that has a pixel value exceeding the initial value. Examplesof conversion from a motion contrast into a pixel value for display inthis processing are shown in FIG. 7A and FIG. 7B. In FIG. 7A, an exampleis shown in which a pixel value for display of zero is assigned to apixel having a motion contrast pixel value equal to or less than thedisplay threshold value and a pixel value for display proportional tothe motion contrast pixel value is assigned to a pixel having a motioncontrast pixel value of from the threshold value to the maximumintensity. In FIG. 7B, an example is shown in which a pixel value fordisplay multiplied by zero is assigned to a motion contrast pixel valueequal to or less than the display threshold value and a pixel value fordisplay multiplied by one is assigned to a motion contrast pixel valuegreater than the threshold value. In any of the examples, a motioncontrast equal to or less than the display threshold value is not usedfor display, and an area having the motion contrast with the pixel valueto be displayed is displayed in a separated manner.

Step S355 illustrated in FIG. 6, which is executed by the displaycontrol portion 149, is a step of displaying the motion contrast imagesubjected to the display threshold processing shown in FIG. 7A and FIG.7B.

Next, Step S356 is a step of changing a display condition. A displaycondition that may be changed is described later. When the displaycondition is changed, the flow returns to Step S354, and an imageupdated under the changed display condition is displayed. When thedisplay condition is not changed, the display information generationprocessing is brought to an end.

Next, a display example of an image generated through the displayinformation generation processing according to this embodiment isillustrated in FIG. 8. In the display screen, an intensity image 400,which is the OCT signal displayed as an image, and a motion contrastimage 401 are displayed side by side. In this display example, a slider402 for operating the display threshold value of the motion contrast isprovided near the motion contrast image 401.

Further, tomographic images 403 in a part of the intensity image 400 aredisplayed near the intensity image 400. The tomographic images aredisplayed in an x-z cross section 403 a and a y-z cross section 403 b.Further, a line 404 indicating the display position of each tomographicimage may be displayed in the intensity image 400 and the motioncontrast image 401. The line 404 indicating the display position may beautomatically determined based on image information under a specificcondition, or may be designated by the inspector. As an example of thespecific condition, any condition in which the tomographic image 403having a macula portion at its center is designated can be used. Asanother example, the inspector may designate an arbitrary position andlength from the intensity image 400 or the motion contrast image 401,and a tomographic image having the designated position and length may bedisplayed.

Further, a slider 405 may be provided, which is used to designate fromeach tomographic image a display range of the intensity image 400 or themotion contrast image 401 in the depth direction. In the slider 405, ona bar 405 b indicating an adjustable range in the depth direction, adepth at which the intensity image 400 or the motion contrast image 401is displayed or the display range in the depth direction is designatedwith a slide portion 405 a. Further, a GUI operation portion 406 foroperating the display position may be provided. Details of the GUIoperation portion 406 are described later. Further, a cursor 407indicating a selected position may be displayed. In this case, it ispreferred that the cursor 407 be displayed as a mark so as tosimultaneously indicate corresponding positions of the intensity image400 and the motion contrast image 401. This display is instructed by thedisplay control portion 149. In the example of FIG. 8, an example inwhich the intensity image 400 and other images are displayed astwo-dimensional planar images is described, but an image to be displayedmay also be a three-dimensional image, an arbitrary tomographic image,or a combination thereof.

In other words, in this example, the intensity image 400 that is thefirst image and the motion contrast image 401 are displayed side byside. Then, in any one of the images, designation of an arbitraryposition in the image is received by the cursor 407 located at thedesignated position, and information on a vessel corresponding to thedesignated position, which is acquired from the motion contrast image401, is superimposed onto the intensity image 400 for display.

Further, in this embodiment, the image illustrated in FIG. 8 as atwo-dimensional plane is generated based on an OCT intensity signalwithout using a fundus camera or a scanning laser ophthalmoscope. Suchan image is referred to as “enface image”. Now, a method of generatingthe enface image is described briefly.

Pieces of information acquired through the A-scan are arrayed from thelight source side in the optical axis direction, and a position at whicha strong signal can be acquired first from a position at which there isno signal is information on a fundus surface layer (retinal surface).Therefore, by linking together reflection intensities of the firststrong signals of the respective A-scan signals at the measurementpositions (imaging positions) on the fundus when the measuring light isscanned along two axes, the enface image of the fundus surface layer forshowing the fundus surface layer two-dimensionally can be acquired.Further, in the A-scan signal, several strong signals corresponding tospecific layers of the retina are found from the first strong signal. Bylinking together reflection intensities of strong signals located at aspecific order counted from the first strong signal, the enface image ata specific layer or depth of the retina can be acquired. Through theabove-mentioned method of acquiring the enface image, the layerstructure in the tomographic image of the fundus Er can be detected fromdata of the three-dimensional tomographic image that is based on theinterference signal set.

Further, the tomographic image acquired through the B-scan isconstructed by continuously acquiring pieces of information through theA-scan along one-axis direction. Accordingly, through acquisition ofcorresponding positions of an image signal of a part of the enface imageand an image signal of a fundus surface layer portion of the acquiredtomographic image, the tomographic image at which position on the enfaceimage the A-scan corresponds to can be detected accurately. When thelayer structure is extracted, the display range of the motion contrastimage, or the like, may be determined based on the layer structure asdescribed later.

Next, a display operation is described. The display operation isillustrated in FIG. 9. In FIG. 9, the intensity image 400 and the motioncontrast image 401 of FIG. 8 are illustrated. As described above, inthis embodiment, the OCT interference signal and the OCTA signal areacquired in the same step, and in the generation of the image to bedisplayed, the position alignment between those signals along the samescan line is finished. Therefore, a correspondence relationship betweenpixels of the intensity image 400 and the motion contrast image 401 isknown in advance.

In the display operation according to this embodiment, when a part ofone of the images (e.g., vessel portion of the intensity image) isselected, a corresponding area of the other image (corresponding vesselportion of the motion contrast image) is superimposed onto the originalimage (intensity image) for display. In an example of FIG. 9, when avessel portion 450 of the intensity image 400 is selected, acorresponding vessel and vessels 450 a connected thereto in the motioncontrast image 401 are superimposed onto corresponding positions of theintensity image 400 for display. More specifically, first, a vessel isselected in one of the images. In this case, each pixel or piece ofpositional information on the vessel is already associated with that ofthe corresponding other image through the position alignment. Under thisstate, positional information on the vessel selected in one of theimages is acquired. Subsequently, based on the acquired positionalinformation, corresponding positional information in the other image isselected. A vessel in the other image is selected as a correspondingvessel based on the selected positional information. Through theabove-mentioned procedure, based on the positional information on thecorresponding vessels in the respective images, the vessel of one of theimages is superimposed onto the other vessel image for display.

It is assumed that a connection state of vessels (thick vessel and agroup of thin vessels branching from the thick vessel) in each of theimages is known in advance through area division processing describedlater.

Therefore, vessels are selected and superimposed in consideration of theconnection state (in a manner in which vessel groups are associated witheach other). As described later, this superimposition for display isperformed by superimposing information (information on the vessel) onone of the intensity image and the motion contrast image, which isselected through input from the cursor 407 or the like, onto an area ofthe other image in which the information is displayed. Thissuperimposition for display is performed by the display control portion149 serving as a display control unit, which is configured to displaythe images superimposed onto each other in the above-mentioned manner onthe display portion 146 serving as a display unit.

As a specific method of selecting the image, as illustrated in FIG. 9, aspecific vessel may be selected, or an area may be designated. An areato be designated may be a rectangular area, or may be a circle having afixed distance from the cursor 407. Further, when the size of an OCTAmeasurement area is smaller than that of an OCT measurement area, theentire motion contrast image may be superimposed. Still further, adisplay color of a vessel of the other image (motion contrast image) tobe superimposed onto the original image may be changed so that whichvessel is superimposed onto the original image (intensity image) fordisplay can be viewed.

Although the example in which the motion contrast image is superimposedonto the intensity image is described above, the intensity image may besuperimposed onto the motion contrast image. In the case when theintensity image is superimposed onto the motion contrast image, when aspecific vessel portion of the motion contrast image 401 is selected, acorresponding vessel and vessels connected thereto in the intensityimage 400 are superimposed onto corresponding positions of the motioncontrast image 401 for display. In this case, it is preferred that, inorder to enhance visibility, a portion corresponding to the vesselportion superimposed for display be expressed by a thick line and a lineobtained by superimposing dotted lines onto one another. As anotherexample, a display color of a vessel to be superimposed may be changed.In general, a vessel that can be identified from the intensity image isa relatively thick vessel, and, hence, understanding of correspondenceto structural information of the intensity image in terms of positionscan be facilitated. Therefore, by superimposing the intensity image ontothe motion contrast image in this manner, a thick vessel can be used asa mark, and comparison with structural information other than a vesselcan be easily performed. Further, through use of the intensity image 400as a basis, not only a vessel, but also, other structural information,e.g., a macula or a lesion, can be displayed together.

As described above, with the configuration in which one of the imageinformation of the OCT intensity image and the image information of theOCTA motion contrast image can be superimposed onto the other imageinformation for display, understanding of correspondence between theintensity image and the motion contrast image can be facilitated.Therefore, an image that is easy to understand intuitively can beprovided.

As an example of display according to this embodiment, display of acursor is described. As illustrated in FIG. 8 and FIG. 9, the cursor 407may be displayed in both of the intensity image 400 and the motioncontrast image 401. The shape of the cursor 407 may be, for example, anarrow as illustrated in the figures, or may be a circle or a rectangle.Further, the cursors 407 of the images may have different shapes. Asanother example, the shape of the cursor 407 may be crosses crossingeach other in the X-axis direction and the Y-axis direction. The cursorsindicate corresponding positions of both images simultaneously. It isdesired that the cursor be freely movable through an operation of amouse (not shown), or the like, attached to the image forming apparatus.Further, a cut position of the tomographic image may be changed inresponse to the movement of the cursor. Through simultaneous display ofcorresponding positions, the images can be easily compared with eachother.

Next, as an example of a change in the display condition according tothis embodiment, a change of the display range and a viewpoint isdescribed. Display magnifications and display positions of the images ora viewpoint position may be changed simultaneously. For example, thedisplay condition is changed through the GUI operation portion 406 ofthe display screen of FIG. 8. Through use of a “zoom” button 406 a, bothimages are enlarged (+) or reduced (−) simultaneously. Further, when theimages are enlarged, through use of a “move” button 406 b, the displayranges of both images are moved in parallel simultaneously. Similarly,through use of a “rotate” button 406 c, display viewpoints of bothimages are rotationally moved simultaneously. The rotational movementcan be applied when both images are displayed three-dimensionally. Inother words, with this configuration, at least one of themagnifications, the displayed positions, and the viewpoint positions ofthe intensity image and the motion contrast image can be changedsimultaneously in both images.

Although the example in which the GUI operation portion 406 includesthree types of buttons is described above, the number of buttons and theshape and type of each button may be different from those of theexample. Another method may be used instead. For example, the displayrange and the viewpoint may be changed through an operation of the mouseattached to the image forming apparatus. The images may be enlarged orreduced by superimposing a cursor of the mouse onto one of the imagesand operating a wheel of the mouse, and the images may be moved (inparallel or rotationally) by moving the mouse while pressing a button ofthe mouse. As an example other than the mouse, in the case of the touchpanel, the image displayed on the panel may be operated by a finger. Inany case, it suffices that the display ranges and the viewpoints of theintensity image and the motion contrast image displayed side by side canbe changed simultaneously. With this configuration, how the image isdisplayed is changed simultaneously in both of the intensity image andthe motion contrast image, and hence, even when how the image isdisplayed is changed, a specific position being viewed can be easilyunderstood.

The display threshold value of the motion contrast pixel value isdescribed. In FIG. 8, the slider 402 for adjusting the threshold valueof the pixel to be displayed is provided. An initial position of thedisplay threshold value may be a representative value set in advance.When the inspector drags the slider with the mouse, the signalprocessing portion 144 determines in Step S356 of FIG. 6 that thedisplay condition is changed (YES). The signal processing portion 144changes the display threshold value, and returns the processing to StepS354 to update the three-dimensional motion contrast image to bedisplayed. At this time, when setting is made so that the threshold canbe adjusted with a relative value with respect to the initial value, anequivalent effect can be obtained even for data of a different object,e.g., a different eye to be inspected or region.

Next, a modified example of the method of displaying vessel informationis described. In this case, a step of acquiring vessel information froma two-dimensional motion contrast image is performed. The step ofacquiring the vessel information may be performed simultaneously withStep S352, which is the smoothing processing step, or may be performedafter Step S352. Specifically, a step of applying the smoothingprocessing data and a step of performing processing of dividing an areacorresponding to a vessel are performed on the motion contrast data.Through execution of those steps of processing, a smooth vessel area canbe extracted.

Next, the area division processing is described. An example of a flow ofthe area division processing is illustrated in FIG. 13.

First, in Step S601, which is a vessel candidate extraction step, thesignal processing portion 144 executes processing of extracting a pixelhaving a pixel value representing the volume data corresponding to themotion contrast of the pixel of interest that is equal to or greaterthan a predetermined threshold value as a vessel candidate.

Next, in Step S602, which is a vessel connection estimation step, thesignal processing portion 144 executes processing of estimating, foreach of the extracted pixels that are the vessel candidates, whether ornot there is a vessel connection relationship, which is a connectionrelationship between those vessels. In the vessel connection estimationstep, for each of the pixels that are the vessel candidates, when anadjacent pixel has a pixel value equal to or greater than thepredetermined threshold value, or when the adjacent pixel is alsoextracted in advance as the vessel candidate, it is determined thatthose pixels are connected vessel candidates. When a specific pixel isdetermined to be a connected vessel candidate in this manner, the rangeof estimation is expanded to a further adjacent pixel. When the pixelvalue of the adjacent pixel is less than the threshold value (or whenthe adjacent pixel is not the vessel candidate),the adjacent pixel isexcluded from the vessel candidate, and another adjacent pixel isestimated. It suffices that the operation of vessel connectionestimation is performed on all or a designated range of the vesselcandidates.

After the estimation of all adjacent pixels of the vessel candidate orall pixels is completed, the signal processing portion 144 performsprocessing of Step S603, which is a vessel recognizing processing step,based on the size of the connected vessel candidate. In the vesselrecognizing processing, the vessel candidate having a predeterminednumber or more of pixel connections is identified as the same relativelythick or main vessel.

Finally, in Step S604, which is a vessel connection relationship datagenerating step, the signal processing portion 144 executes processingof generating first vessel connection relationship data. The firstvessel connection relationship data contains at least information on athree-dimensional distribution of connected vessels and information onthe connection of vessels. For example, volume data containinginformation for identifying, for each pixel, whether or not a vesselexists and identifying each vessel may be used as the first vesselconnection relationship data.

The above-mentioned threshold value for extracting the vessel candidatemay be a value determined in advance, or may be changeable. For example,the OCTA signal of the eye to be inspected is measured, and theinspector may adjust the threshold value after seeing how areas aredivided as vessels. As another example, a representative value may beset as an initial value of the threshold value. When the motion contrastdata is binarized into a vessel area and another area in the areadivision step, the vessel candidate may be extracted through use of aresult of binarization. Similarly, the threshold value of thepredetermined number of connections, which is used for the determinationof a pixel as a vessel, may be a value determined in advance, or may bechangeable. For example, when a plurality of pixels are connected, thosepixels may be determined to be the vessel candidate, and when there isonly a single pixel, the pixel may be excluded from being the candidate.Through execution of such processing, an isolated area can be excludedfrom the motion contrast image, and connected vessels can be extracted.Examples of the isolated area include a local and random change inreflection intensity, e.g., a speckle noise.

Further, although the example in which the intensity image and themotion contrast image have the same size is described, as describedabove, even when the size of a range in which the motion contrast imageis acquired is small, those images can be displayed similarly. Anexample in which the intensity image and the motion contrast image havedifferent sizes is illustrated in FIG. 14A and FIG. 14B. FIG. 14A is anillustration of an example in which the size of the motion contrastimage is changed such that scales of the images match. FIG. 14B is anillustration of an example in which the scale of the motion contrastimage is changed such that the sizes of the images match. In this case,it is preferred that, in one of the images having a wider angle of view,a frame line 416 indicating a display area of the other image bedisplayed. Further, the zoom button 406 a and other buttons illustratedin FIG. 8 may be used to enlarge or reduce the intensity image and themotion contrast image, move the viewpoints of the images, or superimposethe state of one of the images onto the other image.

Next, an example in which a vessel acquired from the intensity image anda vessel acquired from the motion contrast image are compared with eachother is described. An operation performed in this case is describedwith reference to a flowchart of FIG. 15.

First, in Step S651, which is a first vessel connection estimation step,the signal processing portion 144 executes processing of estimating afirst vessel connection relationship from a two-dimensional motioncontrast image. In Step S651, the first vessel connection relationshipcan be estimated by a method similar to the method used in Step S602described above to estimate the connection relationship of the vessels.

Next, in Step S652, which is a second vessel connection estimation step,the signal processing portion 144 executes processing of estimating asecond vessel connection relationship from three-dimensional tomographicimage data. It suffices that the second vessel connection relationshipis estimated for a specific layer. For example, it suffices that, of thethree-dimensional tomographic image data, the three-dimensionaltomographic image data having a depth corresponding to the retinal layeris used to generate a two-dimensional intensity image (e.g., intensityimage 400) to further estimate the second vessel connectionrelationship. In the intensity image, the intensity of a signal acquiredfrom a vessel is relatively weaker than that of a surrounding area dueto, for example, absorption by blood, or the like. In this case, throughselection of the retinal layer, a pixel corresponding to a retinalarterial and venous vessel can be easily extracted as a vessel.

It suffices that division of an area corresponding to a vessel isperformed by a method similar to that of the motion contrast data.Further, it suffices that a value corresponding to the intensity imageis set as the threshold value of a pixel to be used when the pixel isextracted as a vessel. A vessel acquired through the estimation of thesecond vessel connection relationship is information originating fromvascular structure, although its resolution with which a vessel can beextracted is lower than that of the connection relationship of vessels(first vessel connection relationship) acquired from the motion contrastdata.

Next, in Step S653, which is a vessel connection state comparing step,the signal processing portion 144 executes processing of comparingconnection states of vessels having the second vessel connectionrelationship and the first vessel connection relationship. It sufficesthat the comparison between the connection states of vessels isperformed by comparing corresponding positions of the first and secondvessel connection relationships as to whether or not there is a vessel.

Next, in Step S654, which is a comparing result display step (displaystep), the control portion 143 superimposes, for display on the displayportion 146, at least a part of a result of the comparison between theconnection states of vessels onto the intensity image and/or the motioncontrast image. For example, a portion that is not estimated as a vesselin the other image is displayed. Instead, a portion that is estimated asa vessel in both images may be displayed. An area to be subjected to thecomparison may be the entire image, or may be only a specific area.

Superimposition of the vessel image of the motion contrast image ontothe intensity image corresponds to superimposition of information oncapillary vessels that cannot be acquired from the intensity image ontothe intensity image for display. In contrast, superimposition of thevessel image of the intensity image onto the motion contrast imagecorresponds to display of an area in which a blood flow stagnates. Inthe area in which the blood flow stagnates, a motion contrast pixelvalue is less than that of an area in which the blood flows smoothly.When the pixel value is equal to or less than a threshold value, it isdetermined that there is no vessel. Examples of the area in which theblood flow stagnates include a portion in which stagnation occurs due toan aneurysm, a region in which a region having a small internal diameterexists in the middle of a path, and an area in which the vessel isoccluded.

Through the above-mentioned superimposition of information on theconnection state of vessels of one of the images onto the other image,those images can be compared with each other even more easily.

Moreover, in the display of a vessel, a specific vessel may be selectedto be highlighted. In this case, after Step S654, which is the comparingresult display step, a step of selecting a pixel near a specific vesselis further provided. Specifically, for example, an arbitrary pixel isselected from a displayed image with the mouse, or the like. In responseto this selection operation, the signal processing portion 144 selects avessel immediately near the selected pixel. In this case, the selectedvessel has the above-mentioned information on the first and secondvessel connection relationships. For the selected vessel, a result ofcomparing vessels acquired from the intensity image and the motioncontrast image with each other is displayed. It suffices that the resultof comparison is displayed by, for example, superimposing a translucentvessel image having a different color for display. An example in which aspecific vessel is selected and the vessel image of the motion contrastimage is superimposed onto the intensity image is illustrated in FIG. 9.

An aspect having a plurality of modes as a mode of selecting a vesselfor superimposition is conceivable. As the mode in this case, forexample, a mode of superimposing a portion of a vessel designated by thecursor 407 from a papilla to an end portion onto the intensity image 400is conceivable. Further, the plurality of modes may include a mode ofsuperimposing a portion of a vessel designated by the cursor 407 from adesignated position to the end portion onto the intensity image.Further, the plurality of modes may include a mode of designating anarea with the cursor 407 and superimposing a vessel included in thedesignated area onto the intensity image 400. Further, the plurality ofmodes may include a mode of executing estimation of a vascular diameterdescribed later and, at the time of superimposition, not superimposing avessel whose diameter is estimated to be equal to or less than apredetermined vascular diameter. Further, those modes may be combinedwith each other.

With this configuration, a comparison result focusing on a specificvessel can be acquired. For example, one of the retinal arterial andvenous vessels may be selected, and a state of a vessel connected to theselected vessel may be extracted for display. As another example,instead of selecting a specific vessel, the image may be separated intodifferent colors in advance for each extension of a vessel from an opticnerve head. For example, the image may be separated into differentcolors on the basis of directions in which vessels spread from thecenter of the optic nerve head.

A step of acquiring the vessel information from the vessel extractedfrom the motion contrast image may be further provided. For example, atleast a vascular diameter of a vessel area may be detected. The vasculardiameter is obtained by approximating the extracted vessel by a cylinderand estimating an axial direction and a radial direction of thecylinder. Data of the vascular diameter may be further added to, forexample, the volume data of the first vessel connection relationshipdata (information on a three-dimensional distribution of vessels andinformation on the connection of vessels). With this configuration,information on the form of vessels can be acquired from the motioncontrast image.

Further, vessels may be classified based on the calculated vasculardiameter. First, a thin vessel having a diameter smaller than apredetermined diameter is extracted. It suffices that, for example, adiameter of a capillary vessel is set as the predetermined diameter.Then, in a predetermined area of the motion contrast image, a presenceratio between pixels extracted as the thin vessel (e.g., capillaryvessels) and other pixels is calculated. Further, the presence ratio ismapped two-dimensionally for display. With this configuration ofmapping, a density of vessels can be visualized quantitatively. Itsuffices that the predetermined area is divided into areas on the basisof, for example, the macula (or optic nerve head). FIG. 16 is anillustration of an example of mapping. In the example of FIG. 16, a map408 in which a given area is divided into a plurality of areas around amacula portion is superimposed onto a two-dimensional motion contrastimage. Through division into the plurality of areas, a qualitativeevaluation of the density of capillary vessels can be easily performed.Further, mapped information may be superimposed for display not onlyonto the motion contrast image but also onto the two-dimensionalintensity image, or may be displayed on another screen.

With this configuration, information on the form of vessels can bevisualized in association with the intensity image from the motioncontrast image.

(Second Embodiment)

Next, as the second embodiment, a change of the display range in thedepth direction is described.

Through an operation of the slider 405 of FIG. 8, the display ranges tobe displayed in the depth direction of the intensity image and themotion contrast image are changed. The slider 405 includes, as describedabove, the slide portion 405 a indicating the depth range to bedisplayed and the bar 405 b indicating the adjustable range of theretina in the depth direction. The slide portion 405 a enables a changein the depth direction of a width and position to be displayed as animage, and is used for selecting the display range in the depthdirection. In response to the change of the display range through anoperation of the slide portion 405 a, the intensity image 400 and themotion contrast image 401 to be displayed are updated simultaneously.With the configuration according to this embodiment, the display rangesin the depth direction are changed simultaneously, and hence even whenthe range of the displayed depth of an image is changed, a specificposition being viewed can be easily understood.

Although the example in which the slider is used is described above as amethod of changing the depth range, another method may be used. Forexample, a method of substantially changing the displayed depth bydisplaying the image for each layer of the layer structure of the eye tobe inspected may be used. This embodiment includes a step of detecting,from the three-dimensional tomographic image data, layers of the layerstructure of the tomographic image of the eye to be inspected(corresponding to Step S300 of FIG. 5). In a step of setting thedisplayed depth, the displayed depth may be selected based on the layerstructure information on the eye to be inspected instead of the slider.The layer structure of a human eye is known, and includes the followingsix layers, for example. The six layers are (1) a nerve fiber layer(NFL), (2) a layer that is a combination of a ganglion cell layer (GCL)and an inner plexiform layer (IPL), (3) a layer that is a combination ofan inner nuclear layer (INL) and an outer plexiform layer (OPL), (4) alayer that is a combination of an outer nuclear layer (ONL) and anexternal limiting membrane (ELM), (5) a layer that is a combination ofan ellipsoid zone (EZ), an interdigitation zone (IZ), and a retinalpigment epithelium (RPE), and (6) a choroid.

Segmentation of those layers of the retina is described. The mapgeneration portion 148 applies a median filter and a Sobel filter to atomographic image to be processed, which is extracted from the intensityimage, to generate images (hereinafter referred to also as “medianimage” and “Sobel image”, respectively). Next, a profile is generatedfor each A-scan from the generated median image and Sobel image. Aprofile of the luminance value is generated from the median image, and aprofile of a luminance gradient is generated from the Sobel image. Then,a peak within the profile generated from the Sobel image is detected.The profile of the median image corresponding to a portion around thedetected peak and a portion between the peaks is referred to, to therebyextract a border between areas of the retinal layer. The pixel and thelayer are associated with each other based on the extracted borderinformation.

FIG. 10 is an illustration of an example of a method of selecting alayer to be displayed after the segmentation. In this example, selectionbuttons 410 corresponding to the respective layers in the tomographicimage are provided. Through selection of the selection buttons 410, thedisplay range of the image or the layer to be displayed is designatedselectively. One selection button 410 may be provided for each layer asillustrated in FIG. 10, or may be provided for each combination of aplurality of layers. For example, a selection button may be provided foreach of upper layers and lower layers of the retina. As another example,buttons may be provided so that the display range can be selected on aborder-by-border basis. Further, those configurations may be combined.Specifically, radio buttons 411 for switching whether the display rangeis selected on a layer-by-layer basis or on a border-by-border basis maybe provided as illustrated in FIG. 10. Still further, in order tofacilitate understanding of correspondence between the buttons and thelayers or borders, the tomographic image 412 and the selection buttons410 and other buttons may be displayed side by side.

In response to the processing of selecting a layer to be displayed, theintensity image and the motion contrast image are updated. A slider 413for adjusting the display threshold value of the motion contrast imagemay be provided for each layer so that the display threshold value canbe readjusted when the images are updated. With the above-mentionedconfiguration, the layer to be displayed is changed simultaneously forthe intensity image and the motion contrast image, and hence, even whenthe layer to be displayed is changed, a specific layer being viewed canbe easily understood. Further, through a change of the depth range basedon the layer structure, information focusing on a specific layer can bedisplayed.

When the images are displayed based on the layer structure of the eye tobe inspected, the intensity image and the motion contrast image may betwo-dimensional images. Now, a step of generating the two-dimensionalimage is described. In the step of generating the two-dimensional image,based on the designated depth or the selected layer, pieces ofthree-dimensional tomographic image data are projected and integrated inthe depth direction within the corresponding range to generate atwo-dimensional intensity image. An example of a projection method isdescribed with reference to FIG. 11A and FIG. 11B.

FIG. 11A is an illustration of a tomographic image, and a range betweendepths Z1 and Z2 in the depth directions corresponds to a selectedrange. FIG. 11B is an illustration of three-dimensional volume data 500and a depth indication bar 501. A volume area 502 designated as a rangebetween the depths Z1 and Z2 on the depth indication bar 501 is theselected range. A two-dimensional intensity image can be generated byprojecting and integrating voxels within the volume area 502 (the volumedata of the three-dimensional tomographic image) in the depth direction.A three-dimensional motion contrast is similarly processed.Specifically, the voxels within the same area are projected andintegrated, or projected or integrated, in the depth direction, tothereby generate a two-dimensional motion contrast image.

In other words, in the embodiment described here, the first image isgenerated by projecting and integrating pieces of three-dimensionaltomographic image data of the fundus Er within the display range, andthe two-dimensional motion contrast image is generated by projecting andintegrating pieces of volume data of the motion contrast image withinthe display range in the depth direction. It is preferred that thosegenerated images be displayed side by side. Further, at the time ofimage generation for display, it is preferred that, in order to generateany one of the first image and the two-dimensional motion contrastimage, the tomographic image at the position corresponding to thedesignated position be generated for display based on at least any oneof data of the three-dimensional tomographic image and the volume dataof the motion contrast image.

The two-dimensional image can be generated not only by integrating thepixel values of corresponding pixels but also by extracting andprojecting representative values, e.g., maximum values. The generatedtwo-dimensional images are displayed as the intensity image and themotion contrast image. Further, instead of performing the processes ofdisplaying those images simultaneously, those processes may be performedsequentially such that the display threshold value of the motioncontrast image is changed after the two-dimensional motion contrastimage is generated and a display state of the image is examined. Bygenerating two-dimensional images and displaying the images side byside, all of the information focusing on a specific layer can beunderstood at once.

Although the voxels are projected and integrated within the designatedfixed depth range in the example illustrated in FIG. 11A and FIG. 11B,the voxels may be projected and integrated within a depth rangecorresponding to the layer structure. In other words, a depth rangecorresponding to the layer selected for each position may be selected,and the voxels may be projected and integrated within the selectedrange. With this configuration, the two-dimensional image furtherreflecting the layer structure can be generated.

Further, when the above-mentioned two-dimensional image is displayed,the tomographic image may also be displayed side by side as in FIG. 8.In this case, the position of the cross section of the tomographic imageis designated on any one of a two-dimensional intensity image and atwo-dimensional motion contrast image. Further, as the tomographic imageto be displayed, a tomographic image at a position designated from thethree-dimensional tomographic image data and/or the three-dimensionalmotion contrast or at a corresponding position is selected (orgenerated). The original image from which the position of thetomographic image to be displayed is designated and the tomographicimage to be displayed may be combined with each other as necessary.Further, the tomographic image of the motion contrast image may besuperimposed onto the tomographic image of the intensity image fordisplay. Still further, the indication line 404 indicating the crosssection of the tomographic image may be displayed in both images asillustrated in FIG. 8. With the configuration in which the position ofthe tomographic image can be designated for display from thetwo-dimensional image of any one of the intensity image and the motioncontrast image, the tomographic image at a position desired to be viewedcan be easily displayed.

Further, information on the layer structure may also be displayed in theabove-mentioned tomographic image. In this case, based on informationobtained by detecting the layer structure of the tomographic image ofthe eye to be inspected from the three-dimensional tomographic imagedata, the layer structure is superimposed onto the tomographic image fordisplay. The information on the layer structure may be displayed in anymanner as long as each layer of the layer structure can be identified.For example, a line may be displayed on each layer border, or respectivelayers may be displayed in different colors. An example in which thetomographic image is displayed through division into layers isillustrated in FIG. 12. In the example of FIG. 12, border lines 422(dotted lines) based on the detected layers are superimposed onto thetomographic image for display. Border lines 422 a of selected layers aredisplayed as thick dotted lines, and a selected layer area 423 isdisplayed in gray. Through superimposition of the detected informationon the layer structure, more understandable display can be achieved.

(Third Embodiment)

Next, a third embodiment of this disclosure is described. In thisembodiment, a fundus image is acquired separately from acquisition ofthe motion contrast image through OCT. The fundus image may be acquiredwith a fundus camera, a scanning laser ophthalmoscope (SLO), or a vesselimage acquired through fluorescence angiography. This configurationforms, as in the configuration for generating the intensity imagethrough OCT according to the first embodiment, an object-to-be-inspectedimage acquiring unit configured to acquire an object-to-be-inspectedimage within a first predetermined range of the eye to be inspected 118.As in the intensity image acquired through OCT, a third vesselconnection relationship is also calculated from the fundus imageacquired through this configuration. The position alignment is performedbased on the characteristic amounts of vessels between the calculatedvessel connection relationship and the first vessel connectionrelationship acquired from the motion contrast image. A known positionalignment algorithm can be used for the position alignment.

After the position alignment is performed, the images are displayed sideby side. A display example is illustrated in FIG. 17. In the example ofFIG. 17, a motion contrast image (OCTA image) 401 and a fundus image 420are displayed side by side. The fundus image to be displayed is switchedthrough use of a radio switch portion 421. For example, through theswitching operation, the fundus image to be displayed is switched amonga two-dimensional intensity image acquired through integration of OCTdata, the fundus image taken by the fundus camera, and the fundus imagetaken by the SLO. As in the first embodiment, the cursors 407 aredisplayed in both images at corresponding positions.

Further, it suffices that both images can be enlarged or movedsimultaneously as in the first embodiment (display example illustratedin FIG. 8). With this configuration, an image acquired through anotherobservation apparatus can be easily compared with the motion contrastimage. For example, when the motion contrast image and an image acquiredfrom a color fundus camera are displayed side by side, a color vesselphotograph and the motion contrast image can be easily compared witheach other.

(Fourth Embodiment)

Next, a fourth embodiment of this disclosure is described. In thisembodiment, the connection relationship of vessels acquired from themeasured motion contrast image and the connection relationship ofvessels acquired from another motion contrast image are displayed sideby side. It suffices that an image taken previously is used as anothermotion contrast image. A display example according to the fourthembodiment is illustrated in FIG. 18A and FIG. 18B. In the example ofFIG. 18A and FIG. 18B, the motion contrast image 401 on the left side iscompared with a motion contrast image 430 on the right side, which wastaken in the past. As in the third embodiment, the position alignment isperformed between those images based on the connection relationships ofvessels. Further, it suffices that both images can be enlarged or movedsimultaneously. In this case, the operation executed in the step ofacquiring the object-to-be-inspected image in the above-mentionedembodiments is replaced by a step of acquiring another motion contrastimage at a time different from the time of the signal acquisition stepwhen the motion contrast image is measured.

In the example of FIG. 18A, an image acquired when a signal of a vesselin an area 431 of the motion contrast image 401 surrounded by the brokenlines disappears is illustrated as a schematic diagram. Displayedconnection states of both images are compared with each other, and atleast a part of a comparison result is superimposed onto any one of theimages for display. In FIG. 18B, an example in which a comparison resultof the area 431 (vessels) is superimposed for display by the dottedlines is shown. With this configuration, a change with time of theconnection relationship of vessels can be easily understood.

The comparison processing executed in this case may be a comparisonbetween a pair of images, or may be a comparison among a plurality ofimages. When a plurality of images are compared with one another, thoseimages may be displayed in different colors. As another example, animage to be compared may be switched sequentially.

As described above, according to each of the above-mentionedembodiments, information obtained from one of the motion contrast imageacquired through OCTA and the intensity image acquired through OCT isdisplayed in the other image, to thereby enable provision of an imagethat is easy to understand intuitively. In other words, understanding ofcorrespondence between the motion contrast image acquired through OCTAand the intensity image acquired through OCT can be facilitated.Further, when the images are displayed side by side, images that areeasier to be compared with each other can be provided by displaying thecursor indicating corresponding pixels.

Further, through provision of the step in which the magnifications,display positions, and viewpoint positions of both images can be changedsimultaneously, even when how the image is displayed is changed, animage enabling easy understanding of a specific position being viewedcan be provided.

In the above, the preferred embodiments of this disclosure are describedin detail, but this disclosure is not limited to the above-mentionedspecific embodiments, and various modifications and changes may be madewithin the gist of this disclosure described in the appended claims. Forexample, the positional relationship of the displayed images and theshape of the GUI may be changed. Further, the images may be displayedthrough stereoscopic vision with a 3D display.

(Other Embodiments)

This disclosure is not limited to the above-mentioned embodiments, andmay be carried out with various modifications and changes withoutdeparting from the spirit of this disclosure. For example, in theembodiments, a case in which the object to be inspected is an eye isdescribed, but this disclosure is also applicable to an object to beinspected other than an eye, e.g., skin or an organ. In this case, thisdisclosure includes an aspect as an image forming method or apparatusfor forming an image based on data acquired from a medical device otherthan an ophthalmologic apparatus, e.g., an endoscope. Further, it isdesired that the above-mentioned OCT apparatus be understood as aninspection apparatus exemplified by the ophthalmologic apparatus andthat the eye to be inspected be understood as one aspect of the objectto be inspected.

Embodiment(s) of the present invention can also be realized by acomputer of a system or an apparatus that reads out and executescomputer executable instructions (e.g., one or more programs) recordedon a storage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., an application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., a central processingunit (CPU), or a micro processing unit (MPU)) and may include a networkof separate computers or separate processors to read out and to executethe computer executable instructions. The computer executableinstructions may be provided to the computer, for example, from anetwork or the storage medium. The storage medium may include, forexample, one or more of a hard disk, a random-access memory (RAM), aread only memory (ROM), a storage of distributed computing systems, anoptical disk (such as a compact disc (CD), a digital versatile disc(DVD), or a Blu-ray Disc (BD)™), a flash memory device, a memory card,and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2016-009544, filed Jan. 21, 2016, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image display method comprising: acquiring afirst image of a fundus within a first area of an eye to be inspected;acquiring interference signal sets corresponding to a plurality offrames, which are acquired with an intention to acquire the same crosssection, for a plurality of different cross sections; generating, basedon the interference signal sets corresponding to the plurality offrames, a motion contrast image of the fundus within a second areaincluded in the first area; and superimposing, for display, informationacquired from a portion of the motion contrast image of the fundus ontoa corresponding position of the first image of the fundus.
 2. An imagedisplay method according to claim 1, wherein information on a vessel ofthe eye to be inspected, which is acquired from a portion of the motioncontrast image, is superimposed onto a corresponding position of thefirst image for display.
 3. An image display method according to claim2, wherein the information on the vessel of the eye to be inspectedcomprises information indicating a connection state of the vessel.
 4. Animage display method according to claim 1, further comprising displayingthe first image and the motion contrast image side by side.
 5. An imagedisplay method according to claim 4, further comprising receivingdesignation of a position in at least one of the first image and themotion contrast image, wherein information on a vessel of the eye to beinspected corresponding to the designated position, which is acquiredfrom the motion contrast image, is superimposed onto the first image fordisplay.
 6. An image display method according to claim 4, furthercomprising displaying a mark simultaneously indicating correspondingpositions of the first image and the motion contrast image.
 7. An imagedisplay method according to claim 4, further comprising simultaneouslychanging at least one of magnifications, displayed positions, andviewpoint positions of the first image and the motion contrast image. 8.An image display method according to claim 1, wherein the first image isgenerated based on an interference signal set acquired within the firstarea.
 9. An image display method according to claim 8, wherein theinterference signal set used for the generation of the first image isincluded in the interference signal sets corresponding to the pluralityof frames.
 10. An image display method according to claim 8, wherein thefirst image is generated by superimposing, onto one another, pieces ofdata of a three-dimensional tomographic image of the eye to beinspected, which are acquired from the interference signal setscorresponding to the plurality of frames.
 11. An image display methodaccording to claim 8, further comprising performing position alignmentbetween data of a three-dimensional tomographic image of the eye to beinspected that is based on the interference signal set used for thegeneration of the first image and data of a three-dimensionaltomographic image of the eye to be inspected that is based on aninterference signal set used for the generation of the motion contrastimage.
 12. An image display method according to claim 11, furthercomprising changing a display range of the three-dimensional tomographicimage in a depth direction, wherein the motion contrast image to bedisplayed is updated in response to the change of the display range. 13.An image display method according to claim 12, further comprisingdetecting layer structure of a tomographic image of the eye to beinspected from the data of the three-dimensional tomographic image,wherein the display range is selected based on the detected layerstructure.
 14. An image display method according to claim 12, whereinthe first image is generated by projecting and integrating pieces ofdata of the three-dimensional tomographic image of the eye to beinspected within the display range, and wherein the motion contrastimage that is two-dimensional, which is generated by projecting andintegrating pieces of volume data of the motion contrast image in thedepth direction within the display range, and the first image aredisplayed.
 15. An image display method according to claim 14, furthercomprising generating, for display, a tomographic image at a positioncorresponding to a position designated in at least one of the firstimage and the two-dimensional motion contrast image, based on at leastone of the data of the three-dimensional tomographic image and thevolume data of the motion contrast image.
 16. An image display methodaccording to claim 8, wherein the first image comprises a motioncontrast image acquired at a time different from a time when the motioncontrast image is acquired.
 17. An image display method according toclaim 1, further comprising: applying smoothing processing to volumedata of the motion contrast image; and applying area division processingto the volume data to which the smoothing processing has been applied.18. An image display method according to claim 17, further comprising:extracting, in the area division processing, a pixel that is a vesselcandidate based on a pixel value indicating the volume data of themotion contrast image; estimating a pixel having a vessel connectionrelationship with the pixel that is the vessel candidate; andrecognizing, as a vessel, the pixel that is the vessel candidateestimated as having the vessel connection relationship with apredetermined number or more of pixels.
 19. An image display methodaccording to claim 18, further comprising: estimating the vesselconnection relationship from the first image; comparing connectionstates of the vessels with each other based on the vessel connectionrelationship estimated from each of the motion contrast image and thefirst image; and superimposing, for display, information indicating atleast a portion of the connection state of the vessel, which is a resultof the comparison, onto at least one of the first image and the motioncontrast image.
 20. An image display method according to claim 18,further comprising: selecting a pixel near the pixel recognized as thevessel; and displaying a pixel estimated as having the vessel connectionrelationship with the selected near pixel in a manner different from theselected near pixel and a pixel near the pixel.
 21. An image displaymethod according to claim 18, further comprising: comparing a vasculardiameter calculated based on the pixel recognized as the vessel and apredetermined diameter with each other to extract a thin vessel having adiameter smaller than the predetermined diameter; and mapping, fordisplay, a presence ratio between a pixel recognized as the extractedthin vessel and another pixel in a predetermined area of the motioncontrast image.
 22. A non-transitory computer-readable storage mediumhaving stored thereon a program for causing a computer to execute eachof the steps of the image display method of claim
 1. 23. An imagedisplay method according to claim 1, further comprising receivingdesignation of a position in at least one of the first image and themotion contrast image, wherein information on a vessel of the eye to beinspected corresponding to the designated position, which is acquiredfrom the motion contrast image, is superimposed onto the first image fordisplay.
 24. An image display method according to claim 1, furthercomprising changing a display range of a three-dimensional tomographicimage of the eye to be inspected in a depth direction of the eye to beinspected, the three-dimensional tomographic image being acquired fromthe interference signal sets corresponding to the plurality of frames,wherein the motion contrast image to be displayed is updated in responseto the change of the display range.
 25. An image display methodaccording to claim 1, further comprising generating, for display, atomographic image at a position corresponding to a position designatedin at least one of the first image and a two-dimensional motion contrastimage, based on at least one of data of a three-dimensional tomographicimage and volume data of the motion contrast image, the two-dimensionalmotion contrast image being acquired from at least a portion of thevolume data in a depth direction of the eye to be inspected, thethree-dimensional tomographic image being acquired from the interferencesignal sets corresponding to the plurality of frames.
 26. An imagedisplay method according to claim 1, further comprising: comparing avascular diameter and a predetermined diameter with each other toextract a thin vessel having a diameter smaller than the predetermineddiameter, the vascular diameter being calculated by analyzing at leastone vessel in volume data of the motion contrast image; and mapping, fordisplay, a presence ratio between a pixel recognized as the extractedthin vessel and another pixel in a predetermined area of the motioncontrast image.
 27. An image display method according to claim 1,wherein the information is information on a vessel of the eye to beinspected and is acquired by analyzing at least one vessel in volumedata of the motion contrast image.
 28. An image display apparatuscomprising: an image acquiring unit configured to acquire a first imageof a fundus within a first area of an eye to be inspected; a signalacquiring unit configured to acquire interference signal setscorresponding to a plurality of frames, which are acquired with anintention to acquire the same cross section, for a plurality ofdifferent cross sections; a motion contrast image generation unitconfigured to generate, based on the interference signal setscorresponding to the plurality of frames, a motion contrast image of thefundus within a second area included in the first area; and a displaycontrol unit configured to superimpose, for display, informationacquired from a portion of the motion contrast image of the fundus ontoa corresponding position of the first image of the fundus.
 29. An imagedisplay method comprising: acquiring a first image of a fundus within afirst area of an eye to be inspected; acquiring interference signal setscorresponding to a plurality of frames, which are acquired with anintention to acquire the same cross section, for a plurality ofdifferent cross sections; generating, based on the interference signalsets corresponding to the plurality of frames, a motion contrast imageof the fundus within a second area included in the first area; andsuperimposing, for display, information on a vessel of the eye to beinspected, which is extracted from one of the first image and the motioncontrast image of the fundus, onto a corresponding position of anotherone of the first image and the motion contrast image of the fundus. 30.A non-transitory computer-readable storage medium having stored thereona program for causing a computer to execute each of the steps of theimage display method of claim
 29. 31. An image display method accordingto claim 29, further comprising receiving designation of a position inat least one of the first image and the motion contrast image, whereininformation on a vessel of the eye to be inspected corresponding to thedesignated position, which is acquired from the motion contrast image,is superimposed onto the first image for display.
 32. An image displaymethod according to claim 29, further comprising changing a displayrange of a three-dimensional tomographic image of the eye to beinspected in a depth direction, the three-dimensional tomographic imagebeing acquired from the interference signal sets corresponding to theplurality of frames, wherein the motion contrast image to be displayedis updated in response to the change of the display range.
 33. An imagedisplay method according to claim 29, further comprising generating, fordisplay, a tomographic image at a position corresponding to a positiondesignated in at least one of the first image and a two-dimensionalmotion contrast image, based on a three-dimensional tomographic image ofthe eye to be inspected, the three-dimensional tomographic image beingacquired from the interference signal sets corresponding to theplurality of frames, the two-dimensional motion contrast image beingacquired from at least a part of the three-dimensional tomographic imagein a depth direction of the eye to be inspected.
 34. An image displaymethod according to claim 29, further comprising: comparing a vasculardiameter and a predetermined diameter with each other to extract a thinvessel having a diameter smaller than the predetermined diameter, thevascular diameter being calculated by analyzing at least one vessel involume data of the motion contrast image; and mapping, for display, apresence ratio between a pixel recognized as the extracted thin vesseland another pixel in a predetermined area of the motion contrast image.35. An image display method according to claim 29, wherein theinformation on the vessel is acquired by analyzing at least one vesselin volume data of the motion contrast image.
 36. An image displayapparatus comprising: an image acquiring unit configured to acquire afirst image of a fundus within a first area of an eye to be inspected; asignal acquiring unit configured to acquire interference signal setscorresponding to a plurality of frames, which are acquired with anintention to acquire the same cross section, for a plurality ofdifferent cross sections; a motion contrast image generation unitconfigured to generate, based on the interference signal setcorresponding to the plurality of frames, a motion contrast image of thefundus within a second area included in the first area; and a displaycontrol unit configured to superimpose, for display, information on avessel of the eye to be inspected, which is extracted from one of thefirst image and the motion contrast image of the fundus, onto acorresponding position of another one of the first image and the motioncontrast image of the fundus.